Validation of the Advanced Practice Nursing Competency Assessment Instrument in a hospital environment

ABSTRACT Objectives: to evaluate the measurement properties of the Advanced Practice Nursing Competency Assessment Instrument – Brazilian version, in the hospital environment. Methods: a methodological study conducted in a hospital with 238 nurses. Three instruments collect the data: sample characterization form, Brazilian version of the Advanced Practice Nursing Competency Assessment Instrument, and the category “therapeutic interventions” of the nurse competence scale. Construct validity was verified by confirmatory factor analysis and Spearman’s correlation coefficient, and reliability by Cronbach’s Alpha and composite reliability. Results: in the factor analysis, the model converged to a satisfactory result. The study found acceptable evidence of reliability (Cronbach’s Alpha, 0.76-0.87; and composite reliability, 0.85-0.90). Conclusions: the instrument demonstrated evidence of construct validity and internal consistency and can be used in practice


INTRODUCTION
The current public health landscape, especially when considering the region of Latin America and the Caribbean, reveals the detriment of access and universal health coverage to citizens due to the contrasting socio-cultural, geographical, and economic realities and gender inequalities to which they are subjected (1) .The aging of the population, the spread of chronic diseases, the consequences of climate change, as well as internal migration between countries also constitute obstacles to qualified and resolute care concerning the health process-disease (1)(2) .
Faced with increasingly complex health demands and a lack of professionals, mainly due to its poor territorial distribution (2) , the Advanced Nursing Practice (ANP) emerges as a strategy for access to health services, as it consists of the exercise of a nurse with at least a master's degree, integrated into the interprofessional team, capable of making complex decisions and full of clinical skills to develop an expanded role (2)(3) .
The scope of the practice performed by the Advanced Practice Nurse (APN) varies according to the regional context in which it is inserted.However, in general, it includes monitoring and treatment of patients with chronic diseases, diagnosis of less complex clinical conditions, request for tests, and prescription of certain medications, according to protocols and clinical guidelines (3)(4) .
Brazil emerges as a country with great potential to carry out this practice, as it has a range of undergraduate and graduate courses in Nursing and pilot projects that have already been developed (5) .In addition, the Professional Practice Law and the National Primary Care Policy guarantee autonomy and relevance to nurses, basic assumptions for the implementation of the ANP (6)(7) .
More recently, in addition to Primary Care, the insertion of the APN in the hospital environment has been discussed to increase these professionals' autonomy in decisions about managed care.After all, the functions performed by nurses at this level of health care are fragmented concerning the union between management and care (8) .
Thus, it is essential to recognize the importance of the full development of the potential and skills of nurses, accentuating their qualification, rethinking their roles and their professional responsibilities, not only in primary care, but also in the hospital, as a way of attesting to better care outcomes (4,9) .
Therefore, the study highlights the Advanced Practice Nursing (10)(11)(12)(13)(14)(15) Competency Assessment Instrument APNCAI -Brazilian version for its recognized methodological rigor and its application at different levels of healthcare to map the competencies of the APN (14)(15) .
Due to the importance of the APN's work in the hospital context to ensure higher quality and safety in the care offered to patients and because of the scarcity of a validated instrument for this purpose in Brazil, the following question guided the development of this project: does the Brazilian version of the APNCAI demonstrate evidence of validity and reliability when considering the hospital context?
The availability of a tool with certified measurement properties for this level of health care will enable managers to identify the competencies of the nurses with whom they work to subsequently guide the implementation of professional development strategies, which may contribute to the achievement of better results.

OBJECTIVES
To evaluate the measurement properties of the Advanced Practice Nursing Competency Assessment Instrument APNCAI Brazilian version, in the hospital environment.

Ethical aspects
Firstly, the author of the instrument authorized its use for the conduct of this research.The Research Ethics Committee of the institution approved the project which met the ethical recommendations regarding research developed with human beings following resolution 466/2012 of the National Health Council.

Design, period, and place of study
It is a methodological study (16) that evaluated the construct validity (structural and hypothesis testing) and reliability (internal consistency) of the Advanced Practice Nursing Competency Assessment Instrument APNCAI -Brazilian version (15) in the hospital setting.For the description of the research, the study adopted the criteria of the checklist Consensus-Based Standards for the selection of health Measurement Instruments (COSMIN) (17) .The study was conducted between October 2020 and April 2021, in an online and face-to-face format, in a hospital school in the city of Campinas, State of São Paulo (SP), Brazil, which serves patients from the Single Health System (SUS).Its mission is to provide quality assistance to its users, in addition to maintaining a commitment to teaching and research.

Population or sample; criteria of inclusion and exclusion
The sample size calculation was based on the research objective of validating the instrument through the analysis of structural validity.For this, the internationally recommended criterion was adopted as "adequate, " which considers a minimum of a hundred participants and five respondents for each item of the instrument, equivalent to 220 professionals (17) .The study selected participants for convenience, and considered as inclusion criterion: being a nurse, regardless of the sector of activity.Professionals who, despite having agreed to participate, left one or more items of the instrument in blank were excluded from the study.

Study protocol
The collection took place in a hybrid manner.For online collection, the hospital's Nursing Department made available the nurses' emails, to which were sent, in two rounds, the virtual invitations containing a link to access the Informed Consent Form and the instruments.In the face-to-face collection, nurses were approached in their respective sectors, guided about the purpose of the study, and invited to participate.Those who accepted signed the Informed Consent Form, received the printed instruments, and delivered them completed to the researcher on the agreed day.The instruments used for the collection were: a personal and professional characterization form of the sample, the APNCAI -Brazilian version (15) , and the "therapeutic interventions" category of the Nurse Competence Scale (NCS) (18)

Personal and Professional Characterization Form
The form contained personal information (age, gender, and marital status) and professional data (professional training, unit, work shift, duty, time of experience as a nurse, and in the area in which he was currently working, number of jobs) to describe the sample of participants.

Advanced Practice Nursing Competency Assessment Instrument -Brazilian version
The APNCAI -the Brazilian version, aims to evaluate the competence of nurses according to the necessary roles and standards of the ANP through 44 items distributed in eight dimensions: Research and Practice based on evidence; Clinical and Professional Leadership; Professional Autonomy; Interprofessional Relationships and Mentoring; Care Management; Teaching and Professional Education, and Health Promotion (14)(15) .
The composition of the dimensions are the following: Research and Practice Based on Evidence -a total of eight items (1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, and 1.8), which involve the promotion of the relationship between the research and identification of scientific evidence most relevant and clinical and care practice; Clinical and Professional Leadership -four (2.1, 2.2, 2.3, and 2.4), that demonstrate the leadership of advanced practice nurses in promoting quality health care, in addition to counseling and consulting with other professionals; Professional Autonomy -eight items (3.1, 3.2, 3.3, 3.4, 3.5, 3.6, 3.7, and 3.8) concerning the assessment of autonomy in the use of pharmacological and non-pharmacological interventions, clinical diagnosis, referral to other professionals, treatments and therapies; Interprofessional Relationships and Mentoring -six items (4.1, 4.2, 4.3, 4.4, 4.5, and 4.6), which reflect the collaboration and relationship with other health care professionals to improve direct and indirect patient care and be a clinical reference for inexperienced professionals; Management of the Quality -four items (items 5.1, 5.2, 5.3, and 5.4), measuring the skills required for the assessment and the systematic promotion of the quality, effectiveness of practices and advanced care for a whole health-disease; Care Management -six items (6.1, 6.2, 6.3, 6.4, 6.5, and 6.6), which represent the coordination of care across the different levels of care in the health system; Teaching and Professional Education -four items (7.1, 7.2, 7.3, and 7.4) that are connected to the role of the educator regarding the learning of patients and their families, other nurses, students, and health professionals; and Health Promotion -four items (8.1, 8.2, 8.3, and 8.4), which focused attention on the improvement and/or recovery of the user health (14)(15) .
Each item of the dimensions is evaluated by a Likert scale with five points, in which the participant answers how often they perform the described competence in their current job through the following answer options: never (1 point), almost never (2 points), sometimes (3 points), almost always (4 points) and always (5 points).Therefore, the higher the score, the higher the frequency of use of the described competence in professional activities (14)(15) .

Nurse Competence Scale
The NCS evaluated the construct validity through hypothesis testing (18) .Through 73 items distributed in seven categories, this scale aims to assess how often nurses use specific competencies.The present study used only the category "therapeutic interventions" (α = 0.87), which has ten items (39, 40, 41, 42, 43, 44, 45, 46, 47, 48) related to flexible planning of activities by nurses, coordination of the multidisciplinary team, use of institutional protocols, installation of relevant knowledge for care, and decision-making according to specific patient situations -such items reflect, in a certain way, the dimensions of the APNCAI -Brazilian version.The response scale for each item is of the Likert type with four points, ranging from 0 (does not apply to my practice) to 3 (used very often).Thus, the higher the score, the greater the frequency with which the professional uses that competence in their professional activities (18) .

Analysis of results and statistics
The collected data were entered in the Microsoft Excel for Windows ® program and processed by Statistical Analysis Software ® version 9.4 and SmartPLS ® 3.2.1 software.Categorical variables underwent descriptive analysis, and these results were organized into absolute and relative frequency tables.For the continuous variables, the study calculated the measurements of position (mean, median, minimum, and maximum) and dispersion (standard deviation).
The scores for the dimensions of the instruments were obtained by averaging the scores of the participants' responses.In the "therapeutic interventions" category of the NCS, the answer "does not apply to my practice" was not considered in the calculation of the average.
The study applied structural validity through confirmatory factor analysis and hypothesis testing to verify the validity of the construct.The factor analysis was conducted using structural equation models (19) in which the adapted version of the APNCAI for the Brazilian culture was considered a second order variable.
When evaluating the model, the study calculated the values of the average variance extracted (AVE) for each factor, and values greater than 0.5 indicated that the model converges to a satisfactory result (20) .It also analyzed the cross-loads to verify if the factorial load of an item was higher in the factor in which it was initially allocated; and the discriminant validity of the model using the Fornell-Larcker criterion, which verifies if the square roots of the AVE are higher than the correlations between the dimensions (21) .
Regarding the hypothesis test, the study formulated the following hypothesis: the higher the score in the dimensions of the APNCAI -Brazilian version, the higher the score in the category "therapeutic interventions" of the NCS.For this analysis, the research used the Spearman's correlation coefficient (22) .
Reliability was evaluated through internal consistency, and, for this, Cronbach's Alpha coefficient and composite reliability of Validation of the Advanced Practice Nursing Competency Assessment Instrument in a hospital environment Altafini J, Dias FCP, Ferreira TDM, Sastre-Fullana P, São-João TM, Gasparino RC.
(CR) were calculated, in which values equal to or greater than 0.7 were considered acceptable (22) .
For the evaluation of structural validity, the research excluded 55 (23.1%) participants for having left more than 50% of the items unanswered in at least one of the dimensions of the APNCAI -Brazilian version.The study calculated the AVE, CR, and Cronbach's alpha of each dimension with the 183 participants, as can be seen in Table 1.
The factor loads of the items in their respective dimensions and cross-factor loads are presented in Table 2.
The square root of the AVE and the correlations between the constructs are shown in Table 3.
The analysis of internal consistency, evaluated by composite reliability and Cronbach's Alpha, was shown in Table 4.
To evaluate the validity of the convergent construct, the study tested the correlations between each dimension of the APNCAI -Brazilian version and the category "therapeutic interventions" of the NCS (Table 5).

DISCUSSION
The evaluation of constructs using instruments validated with methodological rigor is a significant way to guarantee the safe use of tools in clinical and research practice.There is evidence in the literature that the measurement properties of an instrument can be evaluated through validity, reliability, and responsiveness; and, because they are independent and complementary, studies should adopt more than one measurement, a recommendation employed in the present study (17,22) .
The validity of an instrument reveals its ability to measure what it proposes precisely.In this study, validity was assessed using construct validity: structural and hypothesis testing (20,22) .
The structural construct validity was evaluated using a structural equation model considering the PLS estimation method.It is a methodology whose application is still recent in the scientific literature and presents a smaller volume of publications compared to the traditional method based on the analysis of covariances (CB-SEM).In the latter, the evaluation of the measurement model is called confirmatory factor analysis, while in the PLS method, this evaluation is called composite confirmatory analysis (23) .
Although they present some differences, both methods can be used to confirm a measurement model of measuring instruments that are being developed or adapted (23) .One of the main differences between the two methods is that, in PLS, evaluation of goodness-of-fit measures is not required as is done in the covariance-based method (23) .Between the two, PLS has the advantages of obtaining more accurate estimates in situations where the sample size is small, allowing the construction of more complex models and not having as an assumption that the variables included in the analysis present normal distribution (23) .
In the confirmatory factor analysis, the instrument items demonstrated adequacy to the dimensions insofar as all the steps conducted in the process reached the minimum values established.In the analysis of the AVE, it was possible to see      that the items of the dimensions explained more than 50% of the construct in question, that is, the competence of the APN, demonstrating that the results reflect an acceptable model (23) .When analyzing the cross loads and the Fornell-Larcker criterion, the independence between the constructs was verified, demonstrating that the items are accurate and reflect the concept of the dimensions in which they were allocated (24) .
The results obtained in the analysis of the homogeneity of the items were compared to those of the original study, and it was possible to note that the Brazilian version achieved Cronbach's alpha values slightly lower (0.76-0.87) than the Spanish version (0.81-0.92) (14) , but both the Cronbach's Alpha and the CR of each of the dimensions were higher than the recommended minimum (21) .These small variations may be related to the particularities of the participating sample, the conjuncture and the pandemic moment (25) .It is noteworthy that, in addition to Cronbach's Alpha, CR was also used in this study to evaluate the homogeneity of the items, as authors affirm that this measure is more robust and does not underestimate the internal consistency (16)(17) .
At the end of these evaluations, the study showed that the Brazilian version applied in a hospital environment did not suffer any modification in its structure when compared to the original version (14) .That is, it kept the 44 items distributed in the exact eight dimensions same.
The hypothesis was that the higher the score in the dimensions of the APNCAI -Brazilian version, the higher the score would be in the category "therapeutic interventions" of the AVE; and the evaluation of validity through the hypothesis test showed that the data obtained were significant, demonstrating that the more advanced practice skills the nurse has, the more skills related to therapeutic interventions he performs.This AVE category includes items that highlight the importance of evidence-based updating and the nurse's contribution to the nursing team, multidisciplinary team, and the patient (18) .

Study limitations
A limitation of this study is that the number of exclusions may have impaired the sample size, considering the factor analysis.However, even so, the tool demonstrates evidence of validity and reliability to be used in the hospital environment because the sample size for construct validity by hypothesis testing and reliability analysis was larger than the internationally recommended (17) .
In addition, memory bias and social desirability may have interfered with the results.Social desirability occurs when some respondents provide answers that differ from their actual attitudes, values, or behaviors (26) , and almost half of the studies that use questionnaires shows it (27) .From this perspective, participants change their responses to manage the impression caused or through self-deception (to feel good about themselves), especially when researchers interact with survey participants (26) .In contrast, the hybrid data collection presented here has been frequently used in the current epidemiological context (28) , and it brings potential benefits while helping to minimize the risk of social desirability bias.
Differences between cultures are biases or barriers to the reuse of questionnaires despite a fair share of the validated instruments face problems unless they are developed and used in a very distinct and homogeneous group during a limited period (29) , which would have little practical application in research development.There can be cultural differences not only between countries, but also over time, generations, nations, social classes, ethnic groups, regions, industries, professions, and organizations.In addition to the well-established benefits of using questionnaires, they allow comparisons between studies -a potentially useful fact to advance in areas of study.
As the measurement properties of an instrument are not static and may vary according to the population, mode of administration, and sample size (16)(17) , further studies must reevaluate these properties to confirm them in different contexts.

Contributions to the field of Nursing
This study brings a relevant contribution to Nursing, as it will allow the start of skills mapping of the APN in Brazil.It may be valuable for strengthening the discussions and implementation of this practice in the national territory, mapping nurses' competencies, implementing interventions that develop the autonomy of professionals for safe and interdisciplinary practice.In addition, it can help to improve the quality and safety of care offered to patients who need hospitalization in a hospital environment useful.

CONCLUSIONS
In the sample studied, the Advanced Practice Nursing Competency Assessment Instrument APNCAI -Brazilian version demonstrated evidence of construct validity and internal consistency and can be used in practice to assist in the mapping and, in the future, the implementation of strategies for the development of advanced practice nurses in the hospital context.

of Validation of the Advanced Practice Nursing Competency Assessment Instrument in a hospital environment Altafini
J, Dias FCP, Ferreira TDM, Sastre-Fullana P, São-João TM, Gasparino RC.

Table 1 -
Average variance extracted from the dimensions of the Advanced * APNCAI -Advanced Practice Nursing Competency Assessment Instrument -Brazilian version; † AVE -average variance extracted.

Validation of the Advanced Practice Nursing Competency Assessment Instrument in a hospital environment
Altafini J, Dias FCP, Ferreira TDM, Sastre-Fullana P, São-João TM, Gasparino RC.

Table 3 -
Discriminant validity of the factorial model, according to the Fornell-Larcker criterion, of the Advanced Practice Nursing Competency Assess- * APNCAI -Advanced Practice Nursing Competency Assessment Instrument -Brazilian version.

Table 4 -
Composite reliability and Cronbach's alpha of the dimensions of the Advanced Practice Nursing Competency Assessment Instrument -Brazilian version (N = 183), Campinas, São Paulo, Brazil, 2020-2021

Validation of the Advanced Practice Nursing Competency Assessment Instrument in a hospital environment
Altafini J, Dias FCP, Ferreira TDM, Sastre-Fullana P, São-João TM, Gasparino RC.

Table 5 (concluded) of Validation of the Advanced Practice Nursing Competency Assessment Instrument in a hospital environment
Altafini J, Dias FCP, Ferreira TDM, Sastre-Fullana P, São-João TM, Gasparino RC.